Value model for sending notifications

ABSTRACT

In one embodiment, a method includes sending, through a communications network, several volumes of notifications corresponding to a first notification type to multiple users and several volumes of notifications corresponding to a second notification type to multiple users. The method further determines visitation impacts of the volumes of notifications of the first and second notification types and trains a machine-learning model based on the visitation impacts. The machine-learning model generates an assessment of a likelihood of interaction by a recipient user with each of the notifications.

TECHNICAL FIELD

This disclosure generally relates to communications in an onlinesocial-networking system.

BACKGROUND

A social-networking system, which may include a social-networkingwebsite, may enable its users (such as persons or organizations) tointeract with it and with each other through it. The social-networkingsystem may, with input from a user, create and store in thesocial-networking system a user profile associated with the user. Theuser profile may include demographic information, communication-channelinformation, and information on personal interests of the user. Thesocial-networking system may also, with input from a user, create andstore a record of relationships of the user with other users of thesocial-networking system, as well as provide services (e.g., wall posts,photo-sharing, event organization, messaging, games, or advertisements)to facilitate social interaction between or among users.

The social-networking system may send over one or more networks contentor messages related to its services to a mobile or other computingdevice of a user. A user may also install software applications on amobile or other computing device of the user for accessing a userprofile of the user and other data within the social-networking system.The social-networking system may generate a personalized set of contentobjects to display to a user, such as a newsfeed of aggregated storiesof other users connected to the user.

A mobile computing device—such as a smartphone, tablet computer, orlaptop computer—may include functionality for determining its location,direction, or orientation, such as a GPS receiver, compass, gyroscope,or accelerometer. Such a device may also include functionality forwireless communication, such as BLUETOOTH communication, near-fieldcommunication (NFC), or infrared (IR) communication or communicationwith wireless local area networks (WLANs) or cellular-telephonenetworks. Such a device may also include one or more cameras, scanners,touchscreens, microphones, or speakers. Mobile computing devices mayalso execute software applications, such as games, web browsers, orsocial-networking applications. With social-networking applications,users may connect, communicate, and share information with other usersin their social networks.

The social-networking system may send notifications through acommunications network to a mobile or other computing device of a user.Notifications may be sent through service providers that route thenotifications over one or more networks to the destination devices.These service providers may charge for routing the notifications, forexample, by charging a per message fee.

SUMMARY OF PARTICULAR EMBODIMENTS

In particular embodiments, a notification system may delivernotifications to users based on a value model of the notifications anduser types. The notification system may send various types ofnotifications based on social-networking activity, such as, as anexample and not by way of limitation, new post notifications, new photonotifications, event notifications, etc. In particular embodiments, thenotification system may analyze the impact of sending particular typesof notifications at various volumes. The impact of notifications mayinclude the rate of success of the notifications in causing the user toperform some final action that is the ultimate goal of delivering thenotifications. The impact of the sent notifications may be used to traina machine-learning model that determines and assigns values toparticular types of notifications. The notification-type values may beused to determine a volume of notifications to send for each particulartype. As an example and not by way of limitation, if the notificationsystem determines that a social network post notification has a highervalue than an event notification, the notification system may send ahigher volume of new post notifications to users.

The machine-learning model may also analyze the impacts to determine andassign a value to notification types with respect to particularcharacteristics of users. As an example and not by way of limitation,the machine-learning model may determine that users over 40 years of ageand who have not logged into the social network in more than 30 days aremore likely to respond to SMS notifications, and therefore assign ahigher value to SMS notifications with respect to users 40 years andolder who have not logged in 30 or more days. In this manner, thenotification system may be able to obtain a higher benefit from sendingnotifications while reducing the volume of notifications sent.

The embodiments disclosed above are only examples, and the scope of thisdisclosure is not limited to them. Particular embodiments may includeall, some, or none of the components, elements, features, functions,operations, or steps of the embodiments disclosed above. Embodimentsaccording to the invention are in particular disclosed in the attachedclaims directed to a method, a storage medium, a system and a computerprogram product, wherein any feature mentioned in one claim category,e.g. method, can be claimed in another claim category, e.g. system, aswell. The dependencies or references back in the attached claims arechosen for formal reasons only. However any subject matter resultingfrom a deliberate reference back to any previous claims (in particularmultiple dependencies) can be claimed as well, so that any combinationof claims and the features thereof are disclosed and can be claimedregardless of the dependencies chosen in the attached claims. Thesubject-matter which can be claimed comprises not only the combinationsof features as set out in the attached claims but also any othercombination of features in the claims, wherein each feature mentioned inthe claims can be combined with any other feature or combination ofother features in the claims. Furthermore, any of the embodiments andfeatures described or depicted herein can be claimed in a separate claimand/or in any combination with any embodiment or feature described ordepicted herein or with any of the features of the attached claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example network environment associated with asocial-networking system.

FIG. 2 illustrates an example social graph.

FIG. 3 illustrates an example notification system for sendingnotifications to users.

FIG. 4 illustrates an example curve of a visitation impact of sendingnotifications of a particular type as the volume of notifications sentincreases.

FIG. 5 illustrates an example method for delivering notifications tousers based on a value model of the notification types and user types.

FIG. 6 illustrates an example computer system.

DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 illustrates an example network environment 100 associated with asocial-networking system. Network environment 100 includes a clientsystem 130, a social-networking system 160, and a third-party system 170connected to each other by a network 110. Although FIG. 1 illustrates aparticular arrangement of client system 130, social-networking system160, third-party system 170, and network 110, this disclosurecontemplates any suitable arrangement of client system 130,social-networking system 160, third-party system 170, and network 110.As an example and not by way of limitation, two or more of client system130, social-networking system 160, and third-party system 170 may beconnected to each other directly, bypassing network 110. As anotherexample, two or more of client system 130, social-networking system 160,and third-party system 170 may be physically or logically co-locatedwith each other in whole or in part. Moreover, although FIG. 1illustrates a particular number of client systems 130, social-networkingsystems 160, third-party systems 170, and networks 110, this disclosurecontemplates any suitable number of client systems 130,social-networking systems 160, third-party systems 170, and networks110. As an example and not by way of limitation, network environment 100may include multiple client system 130, social-networking systems 160,third-party systems 170, and networks 110.

This disclosure contemplates any suitable network 110. As an example andnot by way of limitation, one or more portions of network 110 mayinclude an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), a portion of the Internet, a portion of the Public SwitchedTelephone Network (PSTN), a cellular telephone network, or a combinationof two or more of these. Network 110 may include one or more networks110.

Links 150 may connect client system 130, social-networking system 160,and third-party system 170 to communication network 110 or to eachother. This disclosure contemplates any suitable links 150. Inparticular embodiments, one or more links 150 include one or morewireline (such as for example Digital Subscriber Line (DSL) or Data OverCable Service Interface Specification (DOCSIS)), wireless (such as forexample Wi-Fi or Worldwide Interoperability for Microwave Access(WiMAX)), or optical (such as for example Synchronous Optical Network(SONET) or Synchronous Digital Hierarchy (SDH)) links. In particularembodiments, one or more links 150 each include an ad hoc network, anintranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, aportion of the Internet, a portion of the PSTN, a cellulartechnology-based network, a satellite communications technology-basednetwork, another link 150, or a combination of two or more such links150. Links 150 need not necessarily be the same throughout networkenvironment 100. One or more first links 150 may differ in one or morerespects from one or more second links 150.

In particular embodiments, client system 130 may be an electronic deviceincluding hardware, software, or embedded logic components or acombination of two or more such components and capable of carrying outthe appropriate functionalities implemented or supported by clientsystem 130. As an example and not by way of limitation, a client system130 may include a computer system such as a desktop computer, notebookor laptop computer, netbook, a tablet computer, e-book reader, GPSdevice, camera, personal digital assistant (PDA), handheld electronicdevice, cellular telephone, smartphone, augmented/virtual realitydevice, other suitable electronic device, or any suitable combinationthereof. This disclosure contemplates any suitable client systems 130. Aclient system 130 may enable a network user at client system 130 toaccess network 110. A client system 130 may enable its user tocommunicate with other users at other client systems 130.

In particular embodiments, client system 130 may include a web browser132, such as MICROSOFT INTERNET EXPLORER, GOOGLE CHROME or MOZILLAFIREFOX, and may have one or more add-ons, plug-ins, or otherextensions, such as TOOLBAR or YAHOO TOOLBAR. A user at client system130 may enter a Uniform Resource Locator (URL) or other addressdirecting the web browser 132 to a particular server (such as server162, or a server associated with a third-party system 170), and the webbrowser 132 may generate a Hyper Text Transfer Protocol (HTTP) requestand communicate the HTTP request to server. The server may accept theHTTP request and communicate to client system 130 one or more Hyper TextMarkup Language (HTML) files responsive to the HTTP request. Clientsystem 130 may render a webpage based on the HTML files from the serverfor presentation to the user. This disclosure contemplates any suitablewebpage files. As an example and not by way of limitation, webpages mayrender from HTML files, Extensible Hyper Text Markup Language (XHTML)files, or Extensible Markup Language (XML) files, according toparticular needs. Such pages may also execute scripts such as, forexample and without limitation, those written in JAVASCRIPT, JAVA,MICROSOFT SILVERLIGHT, combinations of markup language and scripts suchas AJAX (Asynchronous JAVASCRIPT and XML), and the like. Herein,reference to a webpage encompasses one or more corresponding webpagefiles (which a browser may use to render the webpage) and vice versa,where appropriate.

In particular embodiments, social-networking system 160 may be anetwork-addressable computing system that can host an online socialnetwork. Social-networking system 160 may generate, store, receive, andsend social-networking data, such as, for example, user-profile data,concept-profile data, social-graph information, or other suitable datarelated to the online social network. Social-networking system 160 maybe accessed by the other components of network environment 100 eitherdirectly or via network 110. As an example and not by way of limitation,client system 130 may access social-networking system 160 using a webbrowser 132, or a native application associated with social-networkingsystem 160 (e.g., a mobile social-networking application, a messagingapplication, another suitable application, or any combination thereof)either directly or via network 110. In particular embodiments,social-networking system 160 may include one or more servers 162. Eachserver 162 may be a unitary server or a distributed server spanningmultiple computers or multiple datacenters. Servers 162 may be ofvarious types, such as, for example and without limitation, web server,news server, mail server, message server, advertising server, fileserver, application server, exchange server, database server, proxyserver, another server suitable for performing functions or processesdescribed herein, or any combination thereof. In particular embodiments,each server 162 may include hardware, software, or embedded logiccomponents or a combination of two or more such components for carryingout the appropriate functionalities implemented or supported by server162. In particular embodiments, social-networking system 160 may includeone or more data stores 164. Data stores 164 may be used to storevarious types of information. In particular embodiments, the informationstored in data stores 164 may be organized according to specific datastructures. In particular embodiments, each data store 164 may be arelational, columnar, correlation, or other suitable database. Althoughthis disclosure describes or illustrates particular types of databases,this disclosure contemplates any suitable types of databases. Particularembodiments may provide interfaces that enable a client system 130, asocial-networking system 160, or a third-party system 170 to manage,retrieve, modify, add, or delete, the information stored in data store164.

In particular embodiments, social-networking system 160 may store one ormore social graphs in one or more data stores 164. In particularembodiments, a social graph may include multiple nodes—which may includemultiple user nodes (each corresponding to a particular user) ormultiple concept nodes (each corresponding to a particular concept)—andmultiple edges connecting the nodes. Social-networking system 160 mayprovide users of the online social network the ability to communicateand interact with other users. In particular embodiments, users may jointhe online social network via social-networking system 160 and then addconnections (e.g., relationships) to a number of other users ofsocial-networking system 160 to whom they want to be connected. Herein,the term “friend” may refer to any other user of social-networkingsystem 160 with whom a user has formed a connection, association, orrelationship via social-networking system 160.

In particular embodiments, social-networking system 160 may provideusers with the ability to take actions on various types of items orobjects, supported by social-networking system 160. As an example andnot by way of limitation, the items and objects may include groups orsocial networks to which users of social-networking system 160 maybelong, events or calendar entries in which a user might be interested,computer-based applications that a user may use, transactions that allowusers to buy or sell items via the service, interactions withadvertisements that a user may perform, or other suitable items orobjects. A user may interact with anything that is capable of beingrepresented in social-networking system 160 or by an external system ofthird-party system 170, which is separate from social-networking system160 and coupled to social-networking system 160 via a network 110.

In particular embodiments, social-networking system 160 may be capableof linking a variety of entities. As an example and not by way oflimitation, social-networking system 160 may enable users to interactwith each other as well as receive content from third-party systems 170or other entities, or to allow users to interact with these entitiesthrough an application programming interfaces (API) or othercommunication channels.

In particular embodiments, a third-party system 170 may include one ormore types of servers, one or more data stores, one or more interfaces,including but not limited to APIs, one or more web services, one or morecontent sources, one or more networks, or any other suitable components,e.g., that servers may communicate with. A third-party system 170 may beoperated by a different entity from an entity operatingsocial-networking system 160. In particular embodiments, however,social-networking system 160 and third-party systems 170 may operate inconjunction with each other to provide social-networking services tousers of social-networking system 160 or third-party systems 170. Inthis sense, social-networking system 160 may provide a platform, orbackbone, which other systems, such as third-party systems 170, may useto provide social-networking services and functionality to users acrossthe Internet.

In particular embodiments, a third-party system 170 may include athird-party content object provider. A third-party content objectprovider may include one or more sources of content objects, which maybe communicated to a client system 130. As an example and not by way oflimitation, content objects may include information regarding things oractivities of interest to the user, such as, for example, movie showtimes, movie reviews, restaurant reviews, restaurant menus, productinformation and reviews, or other suitable information. As anotherexample and not by way of limitation, content objects may includeincentive content objects, such as coupons, discount tickets, giftcertificates, or other suitable incentive objects.

In particular embodiments, social-networking system 160 also includesuser-generated content objects, which may enhance a user's interactionswith social-networking system 160. User-generated content may includeanything a user can add, upload, send, or “post” to social-networkingsystem 160. As an example and not by way of limitation, a usercommunicates posts to social-networking system 160 from a client system130. Posts may include data such as status updates or other textualdata, location information, photos, videos, links, music or othersimilar data or media. Content may also be added to social-networkingsystem 160 by a third-party through a “communication channel,” such as anewsfeed or stream.

In particular embodiments, social-networking system 160 may include avariety of servers, sub-systems, programs, modules, logs, and datastores. In particular embodiments, social-networking system 160 mayinclude one or more of the following: a web server, action logger,API-request server, relevance-and-ranking engine, content-objectclassifier, notification controller, action log,third-party-content-object-exposure log, inference module,authorization/privacy server, search module, advertisement-targetingmodule, user-interface module, user-profile store, connection store,third-party content store, or location store. Social-networking system160 may also include suitable components such as network interfaces,security mechanisms, load balancers, failover servers,management-and-network-operations consoles, other suitable components,or any suitable combination thereof. In particular embodiments,social-networking system 160 may include one or more user-profile storesfor storing user profiles. A user profile may include, for example,biographic information, demographic information, behavioral information,social information, or other types of descriptive information, such aswork experience, educational history, hobbies or preferences, interests,affinities, or location. Interest information may include interestsrelated to one or more categories. Categories may be general orspecific. As an example and not by way of limitation, if a user “likes”an article about a brand of shoes the category may be the brand, or thegeneral category of “shoes” or “clothing.” A connection store may beused for storing connection information about users. The connectioninformation may indicate users who have similar or common workexperience, group memberships, hobbies, educational history, or are inany way related or share common attributes. The connection informationmay also include user-defined connections between different users andcontent (both internal and external). A web server may be used forlinking social-networking system 160 to one or more client systems 130or one or more third-party system 170 via network 110. The web servermay include a mail server or other messaging functionality for receivingand routing messages between social-networking system 160 and one ormore client systems 130. An API-request server may allow a third-partysystem 170 to access information from social-networking system 160 bycalling one or more APIs. An action logger may be used to receivecommunications from a web server about a user's actions on or offsocial-networking system 160. In conjunction with the action log, athird-party-content-object log may be maintained of user exposures tothird-party-content objects. A notification controller may provideinformation regarding content objects to a client system 130.Information may be pushed to a client system 130 as notifications, orinformation may be pulled from client system 130 responsive to a requestreceived from client system 130. Authorization servers may be used toenforce one or more privacy settings of the users of social-networkingsystem 160. A privacy setting of a user determines how particularinformation associated with a user can be shared. The authorizationserver may allow users to opt in to or opt out of having their actionslogged by social-networking system 160 or shared with other systems(e.g., third-party system 170), such as, for example, by settingappropriate privacy settings. Third-party-content-object stores may beused to store content objects received from third parties, such as athird-party system 170. Location stores may be used for storing locationinformation received from client systems 130 associated with users.Advertisement-pricing modules may combine social information, thecurrent time, location information, or other suitable information toprovide relevant advertisements, in the form of notifications, to auser.

FIG. 2 illustrates example social graph 200. In particular embodiments,social-networking system 160 may store one or more social graphs 200 inone or more data stores. In particular embodiments, social graph 200 mayinclude multiple nodes—which may include multiple user nodes 202 ormultiple concept nodes 204—and multiple edges 206 connecting the nodes.Example social graph 200 illustrated in FIG. 2 is shown, for didacticpurposes, in a two-dimensional visual map representation. In particularembodiments, a social-networking system 160, client system 130, orthird-party system 170 may access social graph 200 and relatedsocial-graph information for suitable applications. The nodes and edgesof social graph 200 may be stored as data objects, for example, in adata store (such as a social-graph database). Such a data store mayinclude one or more searchable or queryable indexes of nodes or edges ofsocial graph 200.

In particular embodiments, a user node 202 may correspond to a user ofsocial-networking system 160. As an example and not by way oflimitation, a user may be an individual (human user), an entity (e.g.,an enterprise, business, or third-party application), or a group (e.g.,of individuals or entities) that interacts or communicates with or oversocial-networking system 160. In particular embodiments, when a userregisters for an account with social-networking system 160,social-networking system 160 may create a user node 202 corresponding tothe user, and store the user node 202 in one or more data stores. Usersand user nodes 202 described herein may, where appropriate, refer toregistered users and user nodes 202 associated with registered users. Inaddition or as an alternative, users and user nodes 202 described hereinmay, where appropriate, refer to users that have not registered withsocial-networking system 160. In particular embodiments, a user node 202may be associated with information provided by a user or informationgathered by various systems, including social-networking system 160. Asan example and not by way of limitation, a user may provide his or hername, profile picture, contact information, birth date, sex, maritalstatus, family status, employment, education background, preferences,interests, or other demographic information. In particular embodiments,a user node 202 may be associated with one or more data objectscorresponding to information associated with a user. In particularembodiments, a user node 202 may correspond to one or more webpages.

In particular embodiments, a concept node 204 may correspond to aconcept. As an example and not by way of limitation, a concept maycorrespond to a place (such as, for example, a movie theater,restaurant, landmark, or city); a website (such as, for example, awebsite associated with social-network system 160 or a third-partywebsite associated with a web-application server); an entity (such as,for example, a person, business, group, sports team, or celebrity); aresource (such as, for example, an audio file, video file, digitalphoto, text file, structured document, or application) which may belocated within social-networking system 160 or on an external server,such as a web-application server; real or intellectual property (suchas, for example, a sculpture, painting, movie, game, song, idea,photograph, or written work); a game; an activity; an idea or theory; anobject in a augmented/virtual reality environment; another suitableconcept; or two or more such concepts. A concept node 204 may beassociated with information of a concept provided by a user orinformation gathered by various systems, including social-networkingsystem 160. As an example and not by way of limitation, information of aconcept may include a name or a title; one or more images (e.g., animage of the cover page of a book); a location (e.g., an address or ageographical location); a website (which may be associated with a URL);contact information (e.g., a phone number or an email address); othersuitable concept information; or any suitable combination of suchinformation. In particular embodiments, a concept node 204 may beassociated with one or more data objects corresponding to informationassociated with concept node 204. In particular embodiments, a conceptnode 204 may correspond to one or more webpages.

In particular embodiments, a node in social graph 200 may represent orbe represented by a webpage (which may be referred to as a “profilepage”). Profile pages may be hosted by or accessible tosocial-networking system 160. Profile pages may also be hosted onthird-party websites associated with a third-party server 170. As anexample and not by way of limitation, a profile page corresponding to aparticular external webpage may be the particular external webpage andthe profile page may correspond to a particular concept node 204.Profile pages may be viewable by all or a selected subset of otherusers. As an example and not by way of limitation, a user node 202 mayhave a corresponding user-profile page in which the corresponding usermay add content, make declarations, or otherwise express himself orherself. As another example and not by way of limitation, a concept node204 may have a corresponding concept-profile page in which one or moreusers may add content, make declarations, or express themselves,particularly in relation to the concept corresponding to concept node204.

In particular embodiments, a concept node 204 may represent athird-party webpage or resource hosted by a third-party system 170. Thethird-party webpage or resource may include, among other elements,content, a selectable or other icon, or other inter-actable object(which may be implemented, for example, in JavaScript, AJAX, or PHPcodes) representing an action or activity. As an example and not by wayof limitation, a third-party webpage may include a selectable icon suchas “like,” “check-in,” “eat,” “recommend,” or another suitable action oractivity. A user viewing the third-party webpage may perform an actionby selecting one of the icons (e.g., “check-in”), causing a clientsystem 130 to send to social-networking system 160 a message indicatingthe user's action. In response to the message, social-networking system160 may create an edge (e.g., a check-in-type edge) between a user node202 corresponding to the user and a concept node 204 corresponding tothe third-party webpage or resource and store edge 206 in one or moredata stores.

In particular embodiments, a pair of nodes in social graph 200 may beconnected to each other by one or more edges 206. An edge 206 connectinga pair of nodes may represent a relationship between the pair of nodes.In particular embodiments, an edge 206 may include or represent one ormore data objects or attributes corresponding to the relationshipbetween a pair of nodes. As an example and not by way of limitation, afirst user may indicate that a second user is a “friend” of the firstuser. In response to this indication, social-networking system 160 maysend a “friend request” to the second user. If the second user confirmsthe “friend request,” social-networking system 160 may create an edge206 connecting the first user's user node 202 to the second user's usernode 202 in social graph 200 and store edge 206 as social-graphinformation in one or more of data stores 164. In the example of FIG. 2,social graph 200 includes an edge 206 indicating a friend relationbetween user nodes 202 of user “A” and user “B” and an edge indicating afriend relation between user nodes 202 of user “C” and user “B.”Although this disclosure describes or illustrates particular edges 206with particular attributes connecting particular user nodes 202, thisdisclosure contemplates any suitable edges 206 with any suitableattributes connecting user nodes 202. As an example and not by way oflimitation, an edge 206 may represent a friendship, family relationship,business or employment relationship, fan relationship (including, e.g.,liking, etc.), follower relationship, visitor relationship (including,e.g., accessing, viewing, checking-in, sharing, etc.), subscriberrelationship, superior/subordinate relationship, reciprocalrelationship, non-reciprocal relationship, another suitable type ofrelationship, or two or more such relationships. Moreover, although thisdisclosure generally describes nodes as being connected, this disclosurealso describes users or concepts as being connected. Herein, referencesto users or concepts being connected may, where appropriate, refer tothe nodes corresponding to those users or concepts being connected insocial graph 200 by one or more edges 206.

In particular embodiments, an edge 206 between a user node 202 and aconcept node 204 may represent a particular action or activity performedby a user associated with user node 202 toward a concept associated witha concept node 204. As an example and not by way of limitation, asillustrated in FIG. 2, a user may “like,” “attended,” “played,”“listened,” “cooked,” “worked at,” or “watched” a concept, each of whichmay correspond to a edge type or subtype. A concept-profile pagecorresponding to a concept node 204 may include, for example, aselectable “check in” icon (such as, for example, a clickable “check in”icon) or a selectable “add to favorites” icon. Similarly, after a userclicks these icons, social-networking system 160 may create a “favorite”edge or a “check in” edge in response to a user's action correspondingto a respective action. As another example and not by way of limitation,a user (user “C”) may listen to a particular song (“Imagine”) using aparticular application (SPOTIFY, which is an online music application).In this case, social-networking system 160 may create a “listened” edge206 and a “used” edge (as illustrated in FIG. 2) between user nodes 202corresponding to the user and concept nodes 204 corresponding to thesong and application to indicate that the user listened to the song andused the application. Moreover, social-networking system 160 may createa “played” edge 206 (as illustrated in FIG. 2) between concept nodes 204corresponding to the song and the application to indicate that theparticular song was played by the particular application. In this case,“played” edge 206 corresponds to an action performed by an externalapplication (SPOTIFY) on an external audio file (the song “Imagine”).Although this disclosure describes particular edges 206 with particularattributes connecting user nodes 202 and concept nodes 204, thisdisclosure contemplates any suitable edges 206 with any suitableattributes connecting user nodes 202 and concept nodes 204. Moreover,although this disclosure describes edges between a user node 202 and aconcept node 204 representing a single relationship, this disclosurecontemplates edges between a user node 202 and a concept node 204representing one or more relationships. As an example and not by way oflimitation, an edge 206 may represent both that a user likes and hasused at a particular concept. Alternatively, another edge 206 mayrepresent each type of relationship (or multiples of a singlerelationship) between a user node 202 and a concept node 204 (asillustrated in FIG. 2 between user node 202 for user “E” and conceptnode 204 for “SPOTIFY”).

In particular embodiments, social-networking system 160 may create anedge 206 between a user node 202 and a concept node 204 in social graph200. As an example and not by way of limitation, a user viewing aconcept-profile page (such as, for example, by using a web browser or aspecial-purpose application hosted by the user's client system 130) mayindicate that he or she likes the concept represented by the conceptnode 204 by clicking or selecting a “Like” icon, which may cause theuser's client system 130 to send to social-networking system 160 amessage indicating the user's liking of the concept associated with theconcept-profile page. In response to the message, social-networkingsystem 160 may create an edge 206 between user node 202 associated withthe user and concept node 204, as illustrated by “like” edge 206 betweenthe user and concept node 204. In particular embodiments,social-networking system 160 may store an edge 206 in one or more datastores. In particular embodiments, an edge 206 may be automaticallyformed by social-networking system 160 in response to a particular useraction. As an example and not by way of limitation, if a first useruploads a picture, watches a movie, or listens to a song, an edge 206may be formed between user node 202 corresponding to the first user andconcept nodes 204 corresponding to those concepts. Although thisdisclosure describes forming particular edges 206 in particular manners,this disclosure contemplates forming any suitable edges 206 in anysuitable manner.

In particular embodiments, an advertisement may be text (which may beHTML-linked), one or more images (which may be HTML-linked), one or morevideos, audio, other suitable digital object files, a suitablecombination of these, or any other suitable advertisement in anysuitable digital format presented on one or more webpages, in one ormore e-mails, or in connection with search results requested by a user.In addition or as an alternative, an advertisement may be one or moresponsored stories (e.g., a news-feed or ticker item on social-networkingsystem 160). A sponsored story may be a social action by a user (such as“liking” a page, “liking” or commenting on a post on a page, RSVPing toan event associated with a page, voting on a question posted on a page,checking in to a place, using an application or playing a game, or“liking” or sharing a website) that an advertiser promotes, for example,by having the social action presented within a pre-determined area of aprofile page of a user or other page, presented with additionalinformation associated with the advertiser, bumped up or otherwisehighlighted within news feeds or tickers of other users, or otherwisepromoted. The advertiser may pay to have the social action promoted. Asan example and not by way of limitation, advertisements may be includedamong the search results of a search-results page, where sponsoredcontent is promoted over non-sponsored content.

In particular embodiments, an advertisement may be requested for displaywithin social-networking-system webpages, third-party webpages, or otherpages. An advertisement may be displayed in a dedicated portion of apage, such as in a banner area at the top of the page, in a column atthe side of the page, in a GUI of the page, in a pop-up window, in adrop-down menu, in an input field of the page, over the top of contentof the page, or elsewhere with respect to the page. In addition or as analternative, an advertisement may be displayed within an application. Anadvertisement may be displayed within dedicated pages, requiring theuser to interact with or watch the advertisement before the user mayaccess a page or utilize an application. The user may, for example viewthe advertisement through a web browser.

A user may interact with an advertisement in any suitable manner. Theuser may click or otherwise select the advertisement. By selecting theadvertisement, the user may be directed to (or a browser or otherapplication being used by the user) a page associated with theadvertisement. At the page associated with the advertisement, the usermay take additional actions, such as purchasing a product or serviceassociated with the advertisement, receiving information associated withthe advertisement, or subscribing to a newsletter associated with theadvertisement. An advertisement with audio or video may be played byselecting a component of the advertisement (like a “play button”).Alternatively, by selecting the advertisement, social-networking system160 may execute or modify a particular action of the user.

An advertisement may also include social-networking-system functionalitythat a user may interact with. As an example and not by way oflimitation, an advertisement may enable a user to “like” or otherwiseendorse the advertisement by selecting an icon or link associated withendorsement. As another example and not by way of limitation, anadvertisement may enable a user to search (e.g., by executing a query)for content related to the advertiser. Similarly, a user may share theadvertisement with another user (e.g., through social-networking system160) or RSVP (e.g., through social-networking system 160) to an eventassociated with the advertisement. In addition or as an alternative, anadvertisement may include social-networking-system content directed tothe user. As an example and not by way of limitation, an advertisementmay display information about a friend of the user withinsocial-networking system 160 who has taken an action associated with thesubject matter of the advertisement.

An advertisement may be presented or otherwise delivered using plug-insfor web browsers or other applications, iframe elements, news feeds,tickers, notifications (which may include, for example, e-mail, ShortMessage Service (SMS) messages, or notifications), or other means. Anadvertisement may be presented or otherwise delivered to a user on amobile or other computing device of the user.

In particular embodiments, social-networking system 160 may determinethe social-graph affinity (which may be referred to herein as“affinity”) of various social-graph entities for each other. Affinitymay represent the strength of a relationship or level of interestbetween particular objects associated with the online social network,such as users, concepts, content, actions, advertisements, other objectsassociated with the online social network, or any suitable combinationthereof. Affinity may also be determined with respect to objectsassociated with third-party systems 170 or other suitable systems. Anoverall affinity for a social-graph entity for each user, subjectmatter, or type of content may be established. The overall affinity maychange based on continued monitoring of the actions or relationshipsassociated with the social-graph entity. Although this disclosuredescribes determining particular affinities in a particular manner, thisdisclosure contemplates determining any suitable affinities in anysuitable manner.

In particular embodiments, social-networking system 160 may measure orquantify social-graph affinity using an affinity coefficient (which maybe referred to herein as “coefficient”). The coefficient may representor quantify the strength of a relationship between particular objectsassociated with the online social network. The coefficient may alsorepresent a probability or function that measures a predictedprobability that a user will perform a particular action based on theuser's interest in the action. In this way, a user's future actions maybe predicted based on the user's prior actions, where the coefficientmay be calculated at least in part on the history of the user's actions.Coefficients may be used to predict any number of actions, which may bewithin or outside of the online social network. As an example and not byway of limitation, these actions may include various types ofcommunications, such as sending messages, posting content, or commentingon content; various types of a observation actions, such as accessing orviewing profile pages, media, or other suitable content; various typesof coincidence information about two or more social-graph entities, suchas being in the same group, tagged in the same photograph, checked-in atthe same location, or attending the same event; or other suitableactions. Although this disclosure describes measuring affinity in aparticular manner, this disclosure contemplates measuring affinity inany suitable manner.

In particular embodiments, social-networking system 160 may use avariety of factors to calculate a coefficient. These factors mayinclude, for example, user actions, types of relationships betweenobjects, location information, other suitable factors, or anycombination thereof. In particular embodiments, different factors may beweighted differently when calculating the coefficient. The weights foreach factor may be static or the weights may change according to, forexample, the user, the type of relationship, the type of action, theuser's location, and so forth. Ratings for the factors may be combinedaccording to their weights to determine an overall coefficient for theuser. As an example and not by way of limitation, particular useractions may be assigned both a rating and a weight while a relationshipassociated with the particular user action is assigned a rating and acorrelating weight (e.g., so the weights total 100%). To calculate thecoefficient of a user towards a particular object, the rating assignedto the user's actions may comprise, for example, 60% of the overallcoefficient, while the relationship between the user and the object maycomprise 40% of the overall coefficient. In particular embodiments, thesocial-networking system 160 may consider a variety of variables whendetermining weights for various factors used to calculate a coefficient,such as, for example, the time since information was accessed, decayfactors, frequency of access, relationship to information orrelationship to the object about which information was accessed,relationship to social-graph entities connected to the object, short- orlong-term averages of user actions, user feedback, other suitablevariables, or any combination thereof. As an example and not by way oflimitation, a coefficient may include a decay factor that causes thestrength of the signal provided by particular actions to decay withtime, such that more recent actions are more relevant when calculatingthe coefficient. The ratings and weights may be continuously updatedbased on continued tracking of the actions upon which the coefficient isbased. Any type of process or algorithm may be employed for assigning,combining, averaging, and so forth the ratings for each factor and theweights assigned to the factors. In particular embodiments,social-networking system 160 may determine coefficients usingmachine-learning algorithms trained on historical actions and past userresponses, or data farmed from users by exposing them to various optionsand measuring responses. Although this disclosure describes calculatingcoefficients in a particular manner, this disclosure contemplatescalculating coefficients in any suitable manner.

In particular embodiments, social-networking system 160 may calculate acoefficient based on a user's actions. Social-networking system 160 maymonitor such actions on the online social network, on a third-partysystem 170, on other suitable systems, or any combination thereof. Anysuitable type of user actions may be tracked or monitored. Typical useractions include viewing profile pages, creating or posting content,interacting with content, tagging or being tagged in images, joininggroups, listing and confirming attendance at events, checking-in atlocations, liking particular pages, creating pages, and performing othertasks that facilitate social action. In particular embodiments,social-networking system 160 may calculate a coefficient based on theuser's actions with particular types of content. The content may beassociated with the online social network, a third-party system 170, oranother suitable system. The content may include users, profile pages,posts, news stories, headlines, instant messages, chat roomconversations, emails, advertisements, pictures, video, music, othersuitable objects, or any combination thereof. Social-networking system160 may analyze a user's actions to determine whether one or more of theactions indicate an affinity for subject matter, content, other users,and so forth. As an example and not by way of limitation, if a user maymake frequently posts content related to “coffee” or variants thereof,social-networking system 160 may determine the user has a highcoefficient with respect to the concept “coffee”. Particular actions ortypes of actions may be assigned a higher weight and/or rating thanother actions, which may affect the overall calculated coefficient. Asan example and not by way of limitation, if a first user emails a seconduser, the weight or the rating for the action may be higher than if thefirst user simply views the user-profile page for the second user.

In particular embodiments, social-networking system 160 may calculate acoefficient based on the type of relationship between particularobjects. Referencing the social graph 200, social-networking system 160may analyze the number and/or type of edges 206 connecting particularuser nodes 202 and concept nodes 204 when calculating a coefficient. Asan example and not by way of limitation, user nodes 202 that areconnected by a spouse-type edge (representing that the two users aremarried) may be assigned a higher coefficient than a user nodes 202 thatare connected by a friend-type edge. In other words, depending upon theweights assigned to the actions and relationships for the particularuser, the overall affinity may be determined to be higher for contentabout the user's spouse than for content about the user's friend. Inparticular embodiments, the relationships a user has with another objectmay affect the weights and/or the ratings of the user's actions withrespect to calculating the coefficient for that object. As an exampleand not by way of limitation, if a user is tagged in first photo, butmerely likes a second photo, social-networking system 160 may determinethat the user has a higher coefficient with respect to the first photothan the second photo because having a tagged-in-type relationship withcontent may be assigned a higher weight and/or rating than having alike-type relationship with content. In particular embodiments,social-networking system 160 may calculate a coefficient for a firstuser based on the relationship one or more second users have with aparticular object. In other words, the connections and coefficientsother users have with an object may affect the first user's coefficientfor the object. As an example and not by way of limitation, if a firstuser is connected to or has a high coefficient for one or more secondusers, and those second users are connected to or have a highcoefficient for a particular object, social-networking system 160 maydetermine that the first user should also have a relatively highcoefficient for the particular object. In particular embodiments, thecoefficient may be based on the degree of separation between particularobjects. The lower coefficient may represent the decreasing likelihoodthat the first user will share an interest in content objects of theuser that is indirectly connected to the first user in the social graph200. As an example and not by way of limitation, social-graph entitiesthat are closer in the social graph 200 (i.e., fewer degrees ofseparation) may have a higher coefficient than entities that are furtherapart in the social graph 200.

In particular embodiments, social-networking system 160 may calculate acoefficient based on location information. Objects that aregeographically closer to each other may be considered to be more relatedor of more interest to each other than more distant objects. Inparticular embodiments, the coefficient of a user towards a particularobject may be based on the proximity of the object's location to acurrent location associated with the user (or the location of a clientsystem 130 of the user). A first user may be more interested in otherusers or concepts that are closer to the first user. As an example andnot by way of limitation, if a user is one mile from an airport and twomiles from a gas station, social-networking system 160 may determinethat the user has a higher coefficient for the airport than the gasstation based on the proximity of the airport to the user.

In particular embodiments, social-networking system 160 may performparticular actions with respect to a user based on coefficientinformation. Coefficients may be used to predict whether a user willperform a particular action based on the user's interest in the action.A coefficient may be used when generating or presenting any type ofobjects to a user, such as advertisements, search results, news stories,media, messages, notifications, or other suitable objects. Thecoefficient may also be utilized to rank and order such objects, asappropriate. In this way, social-networking system 160 may provideinformation that is relevant to user's interests and currentcircumstances, increasing the likelihood that they will find suchinformation of interest. In particular embodiments, social-networkingsystem 160 may generate content based on coefficient information.Content objects may be provided or selected based on coefficientsspecific to a user. As an example and not by way of limitation, thecoefficient may be used to generate media for the user, where the usermay be presented with media for which the user has a high overallcoefficient with respect to the media object. As another example and notby way of limitation, the coefficient may be used to generateadvertisements for the user, where the user may be presented withadvertisements for which the user has a high overall coefficient withrespect to the advertised object. In particular embodiments,social-networking system 160 may generate search results based oncoefficient information. Search results for a particular user may bescored or ranked based on the coefficient associated with the searchresults with respect to the querying user. As an example and not by wayof limitation, search results corresponding to objects with highercoefficients may be ranked higher on a search-results page than resultscorresponding to objects having lower coefficients.

In particular embodiments, social-networking system 160 may calculate acoefficient in response to a request for a coefficient from a particularsystem or process. To predict the likely actions a user may take (or maybe the subject of) in a given situation, any process may request acalculated coefficient for a user. The request may also include a set ofweights to use for various factors used to calculate the coefficient.This request may come from a process running on the online socialnetwork, from a third-party system 170 (e.g., via an API or othercommunication channel), or from another suitable system. In response tothe request, social-networking system 160 may calculate the coefficient(or access the coefficient information if it has previously beencalculated and stored). In particular embodiments, social-networkingsystem 160 may measure an affinity with respect to a particular process.Different processes (both internal and external to the online socialnetwork) may request a coefficient for a particular object or set ofobjects. Social-networking system 160 may provide a measure of affinitythat is relevant to the particular process that requested the measure ofaffinity. In this way, each process receives a measure of affinity thatis tailored for the different context in which the process will use themeasure of affinity.

In connection with social-graph affinity and affinity coefficients,particular embodiments may utilize one or more systems, components,elements, functions, methods, operations, or steps disclosed in U.S.patent application Ser. No. 11/503,093, filed 11 Aug. 2006, U.S. patentapplication Ser. No. 12/977,027, filed 22 Dec. 2010, U.S. patentapplication Ser. No. 12/978,265, filed 23 Dec. 2010, and U.S. patentapplication Ser. No. 13/632,869, filed 1 Oct. 2012, each of which isincorporated by reference. In particular embodiments, one or more of thecontent objects of the online social network may be associated with aprivacy setting. The privacy settings (or “access settings”) for anobject may be stored in any suitable manner, such as, for example, inassociation with the object, in an index on an authorization server, inanother suitable manner, or any combination thereof. A privacy settingof an object may specify how the object (or particular informationassociated with an object) can be accessed (e.g., viewed or shared)using the online social network. Where the privacy settings for anobject allow a particular user to access that object, the object may bedescribed as being “visible” with respect to that user. As an exampleand not by way of limitation, a user of the online social network mayspecify privacy settings for a user-profile page identify a set of usersthat may access the work experience information on the user-profilepage, thus excluding other users from accessing the information. Inparticular embodiments, the privacy settings may specify a “blockedlist” of users that should not be allowed to access certain informationassociated with the object. In other words, the blocked list may specifyone or more users or entities for which an object is not visible. As anexample and not by way of limitation, a user may specify a set of usersthat may not access photos albums associated with the user, thusexcluding those users from accessing the photo albums (while alsopossibly allowing certain users not within the set of users to accessthe photo albums). In particular embodiments, privacy settings may beassociated with particular social-graph elements. Privacy settings of asocial-graph element, such as a node or an edge, may specify how thesocial-graph element, information associated with the social-graphelement, or content objects associated with the social-graph element canbe accessed using the online social network.

In particular embodiments, a notification system may delivernotifications to users based on a value model of the notifications anduser types. Such notifications may be sent through one or more deliverychannels, e.g., sent by one or more communication media (e.g., SMS, MMS,email, application, push, voice) to one or more unique endpoints (e.g.,a telephone number, an email address, a particular client device asspecified by a unique device identifier). The notification system maysend various types of notifications based on social-networking activity.As an example and not by way of limitation, notification types mayinclude new post notifications, new photo notifications, eventnotifications, nearby-friends notifications, friend requestnotifications, user mention notifications, comment notification, phototag notifications, birthday notifications, etc.

In particular embodiments, the notification system may analyze theimpact of sending particular types of notifications at various volumes.A volume of notifications may be, as an example and not by way oflimitation, a number of notifications or a budget for notifications(e.g., the amount of notifications that can be sent for $1,000). Theimpact of notifications may include the rate of success of thenotifications in causing the user to perform some final action that isthe ultimate goal of delivering the notifications—the conversion rate(e.g., completing an action, such as a registration, contentconsumption, a purchase, or visiting the social-networking system). Asan example and not by way of limitation, the impact may include a numberof daily, weekly, or monthly active users of the social-networkingsystem, as a metric of how many users have logged into thesocial-networking system in the relevant time period. The impact of thesent notifications may be used to train a machine-learning model thatdetermines and assigns values to particular types of notifications. Thenotification-type values may be used to determine a volume ofnotifications to send for each particular type. As an example and not byway of limitation, if the notification system determines that a new postnotification has a higher value than an event notification, thenotification system may send a higher volume of new post notificationsto users. The machine-learning model may also analyze the impacts todetermine and assign a value to notification types with respect toparticular characteristics of users. As an example and not by way oflimitation, the machine-learning model may determine that users over 40years of age and who have not logged into the social network in morethan 30 days are more likely to respond to SMS notifications, andtherefore assign a higher value to SMS notifications with respect tousers 40 years and older who have not logged in 30 or more days. In thismanner, the notification system may be able to obtain a higher benefitfrom sending notifications while reducing the volume of notificationssent.

FIG. 3 illustrates a notification system 320 according to an exampleembodiment. Notification system 320 may receive notifications 312 from anotification provider 310, and decide which of the notifications aresent to users. Notification provider 310 may include thesocial-networking system 160 or a third-party system. Notificationsystem 320 may send the notifications 312 that are to be sent to theusers to notification delivery service 340. Notification system 320 mayalso receive information 314 regarding user interaction with thenotifications from interaction handling service 350, and use it to trainmachine-learning model 322.

In particular embodiment, notification system 320 may send variousvolumes of notifications of different types to assess their impact asthe volume grows. As an example and not by way of limitation,notification system 320 may receive 100,000 new post notifications fromthe notification provider 310 and select 5% (5,000) of the notificationsto send to the users and measure their visitation impact. Thenotification system 320 may send the next 5% at a time, and measure thevisitation impact at each volume in order to generate a visitationimpact over volume curve, as illustrated in the example of FIG. 4. Inparticular embodiments, the notification system sends messages indecreasing order of value, for example, sending the 5% most valuablemessages first, then the next 5% most valuable messages, and so on. Theexample curve may reflect the visitation impact of sending notificationsof a particular type as the volume of notifications sent increases. Asan example and not by way of limitation, the benefit of sendingadditional notifications decreases after a certain volume is sent. Forexample, once the most valuable notifications are sent, the remainingnotifications may be less likely cause a conversion. As such, thenotification system may in the future decide to send only up to aparticular volume of notifications of each type.

In particular embodiments, notification system 320 may gather visitationimpact information for multiple notification types as explained above.The notification system 320 may then pass this information tomachine-learning model 322 to integrate the new information into themodel. Machine-learning model 322 may be any type of machine-learningmodel suitable to determine and assign values to notification typesbased on visitation impact, such as, by way of example and notlimitation, decision tree learning, association rule learning,artificial neural network, deep learning, inductive logic programming,support vector machines, etc. The machine-learning model may also usetechniques, methods, and systems disclosed in U.S. patent applicationSer. No. 14/567,218, filed on 11 Dec. 2014, which is incorporated hereinby reference in its entirety.

In particular embodiments, the machine-learning model 322 may analyze acontinuously growing data set to generate values of the notificationtypes. The machine-learning model's data set may include any relevantinformation in the social-networking system. As an example and not byway of limitation, the data set may include information about visitationimpacts, user profile, user demographics, user social data, user usagepatterns, user device type, a user friend count, a user's social-networkage (e.g., time the user has been part of the social-network), anotification time, etc. The generated values may reflect an assessmentof a likelihood of interaction by a recipient of the notification withthe notifications of the type.

In particular embodiments, notification system 320 may determinecharacteristics, categories, or buckets of users that comport to aparticular impact behavior in response to certain notification types. Inparticular embodiments, machine-learning model 322 may detect a largedifference in visitation impacts between a group of users that have aparticular characteristic and users that don't have that characteristic.As an example and not by way of limitation, notification system 320 maydetect that users over 40 years of age are more responsive to postnotifications than other users.

In particular embodiments, the machine-learning model 322 may furthergenerate characteristic values of notification types in relation to usercharacteristics. A characteristic value may reflect an assessment of alikelihood of interaction by a recipient having the characteristic withthe notifications of the type. As an example and not by way oflimitation, an over-40 characteristic value for event notifications mayreflect the likelihood that a user over 40 years of age interacts withan event notification. In particular embodiments, the characteristicvalue may be determined based on a contribution of the characteristic tothe overall visitation impact of the notification type. As an exampleand not by way of limitation, if 75% of the visits responsive to eventnotifications came from users over 40 years of age, the value of theover-40 characteristic may be higher for the event notification typethan the under-40 characteristic. In particular embodiments,machine-learning model 322 may determine the characteristic values ofnotification types of a particular characteristic by sendingnotifications to a group of users but omitting users with the particularcharacteristic, measuring the difference in visitation impacts betweenthe that group of users and groups that contained the characteristic. Inparticular embodiments, the notification type values and thecharacteristic values may be determined on a per country basis or acommunications carrier (e.g., mobile carrier, SMS carrier) basis.

In particular embodiments, notification system 320 may use thenotification type value, the characteristic type value, or both valuesto determine a volume of a message type to send to users having variouscharacteristics. As an example and not by way of limitation,notification system 320 may determine a volume of event notifications tosend during the day based on the value of event notifications. As anexample and not by way of limitation, notification system 320 maydetermine a volume of event notifications to send to users over 40 yearsof age during the day based on the value of event notifications and thecharacteristic value of event notifications for users over 40. Inparticular embodiments, the notification system 320 determines thevolume periodically based on updated data in the machine-learning model322. As an example and not by way of limitation, notification system 320may determine the volumes to be sent each day at the beginning of theday. Notification system 320 may then send the determined volumes ofmessages to the corresponding users. In particular embodiments, theanalyses of the values of notifications types and characteristics may berepeated periodically in an automated manner.

In particular embodiments, notification system 320 may determine thevolume of notifications to send by analyzing the visitation impact overvolume curves. As mentioned above, notification system 320 may sendvarious volumes of notifications of different types to generate curvesof visitation impact over volume for each notification type. Inparticular embodiments, notification system 320 may determine a pointwhere the slope of the curve decreases below a threshold. The thresholdslope may reflect a minimum desirable benefit per additional message,where the notification system 320 should not incur additional sendingcosts. As an example and not by way of limitation, and as shown in thecurve of FIG. 4, when the curve flattens out (i.e., the slope decreases)the additional benefit per volume decreases. In the example of FIG. 4,the point 410 in the curve may be where the slope drops below thethreshold, and would result in determined volume of notifications to bevolume 420. In particular embodiments, the threshold slope may be chosenbased on a limit of notifications that the social-networking system 160may want to send during a period. As an example and not by way oflimitation, the notifications may be SMS messages cost a certain amountof money to send, and the notification system 320 may have a fixed limitof notifications to send per day based on a budget. As such, the slopethresholds may be configured to maximize the visitation impact for afixed amount of notifications to be sent. In particular embodiments,notification system 320 may send the highest valued notifications up toa fixed volume.

In particular embodiments, notification provider 310 may providenotification system 320 a plurality of notifications and thenotification system may select a subset of the notifications to send tothe corresponding recipients. The notifications generated bynotification system 320 may include all notifications associated withsocial-networking activities in a time period. Notification system 320may narrow down the number of notifications that are sent based on thelikelihood of interaction by the recipients, as determined by themachine-learning model 322, and reflected in the generatednotification-type values of the notifications and the characteristictype values of the recipients. A subset of highest-valued notificationsmay yield a desired benefit (e.g., number of conversion, number ofvisits, etc.) while minimizing the amount of notifications transmitted.As an example and not by way of limitation, notification system 320 mayreceive 100,000 notifications from notification provider 310 directed toa group of users (e.g., 100,000 users assuming all notifications aredirected at different users), and based on the notification-type valuesof the notifications and the characteristic type values of therecipients may determine a subset of 40,000 of the notifications forsending to a corresponding subset of the group of users (e.g., 40,000users assuming all notifications are directed at different users). Forexample, the subset of notifications could include the 40,000 highestvalued notifications. As another example and not by way of limitation,the social-networking system may receive 100,000 notifications directedto 50,000 users, and select a subset of 30,000 notifications directed to30,000 users, where the notification system 320 selects the recipientsbased on a desired increase in monthly active users of thesocial-networking system. For example, subsets of recipients andnotifications selected may be based on the user characteristics that aremore likely to respond to the notification type being sent, according tothe values generated by machine-learning model 322.

FIG. 5 illustrates an example method 500 for delivering notifications tousers based on a value model of the notification types and user types.The method may begin at step 510, where the notification system 320 maysend, through a communications network, a plurality of volumes ofnotifications corresponding to a first notification type to a pluralityof users. At step 520, the notification system 320 may send, through thecommunications network, a plurality of volumes of notificationscorresponding to a second notification type to a plurality of users. Atstep 530, the notification system 320 may determine visitation impactsof the volumes of notifications of the first and second notificationtypes. At step 540, the notification system 320 may train amachine-learning model based on the visitation impacts, wherein themachine-learning model generates an assessment of a likelihood ofinteraction by a recipient user with each of the notifications. At step550, the notification system 320 may send, through the communicationsnetwork, a first volume of notifications of the first notification typeto a number of first users, the first volume based on a first type valuefor the first notification type, the first type value obtained from themachine-learning model. At step 560, the notification system 320 maysend, through the communications network, a second volume ofnotifications of the second notification type to a number of secondusers, the second volume based on a second type value for the secondnotification type, the second type value obtained from themachine-learning model. Particular embodiments may repeat one or moresteps of the method of FIG. 5, where appropriate. Although thisdisclosure describes and illustrates particular steps of the method ofFIG. 5 as occurring in a particular order, this disclosure contemplatesany suitable steps of the method of FIG. 5 occurring in any suitableorder. Moreover, although this disclosure describes and illustrates anexample method for delivering notifications to users based on a valuemodel of the notification types and user types including the particularsteps of the method of FIG. 5, this disclosure contemplates any suitablemethod for delivering notifications to users based on a value model ofthe notification types and user types including any suitable steps,which may include all, some, or none of the steps of the method of FIG.5, where appropriate. Furthermore, although this disclosure describesand illustrates particular components, devices, or systems carrying outparticular steps of the method of FIG. 5, this disclosure contemplatesany suitable combination of any suitable components, devices, or systemscarrying out any suitable steps of the method of FIG. 5.

FIG. 6 illustrates an example computer system 600. In particularembodiments, one or more computer systems 600 perform one or more stepsof one or more methods described or illustrated herein. In particularembodiments, one or more computer systems 600 provide functionalitydescribed or illustrated herein. In particular embodiments, softwarerunning on one or more computer systems 600 performs one or more stepsof one or more methods described or illustrated herein or providesfunctionality described or illustrated herein. Particular embodimentsinclude one or more portions of one or more computer systems 600.Herein, reference to a computer system may encompass a computing device,and vice versa, where appropriate. Moreover, reference to a computersystem may encompass one or more computer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems600. This disclosure contemplates computer system 600 taking anysuitable physical form. As example and not by way of limitation,computer system 600 may be an embedded computer system, a system-on-chip(SOC), a single-board computer system (SBC) (such as, for example, acomputer-on-module (COM) or system-on-module (SOM)), a desktop computersystem, a laptop or notebook computer system, an interactive kiosk, amainframe, a mesh of computer systems, a mobile telephone, a personaldigital assistant (PDA), a server, a tablet computer system, anaugmented/virtual reality device, or a combination of two or more ofthese. Where appropriate, computer system 600 may include one or morecomputer systems 600; be unitary or distributed; span multiplelocations; span multiple machines; span multiple data centers; or residein a cloud, which may include one or more cloud components in one ormore networks. Where appropriate, one or more computer systems 600 mayperform without substantial spatial or temporal limitation one or moresteps of one or more methods described or illustrated herein. As anexample and not by way of limitation, one or more computer systems 600may perform in real time or in batch mode one or more steps of one ormore methods described or illustrated herein. One or more computersystems 600 may perform at different times or at different locations oneor more steps of one or more methods described or illustrated herein,where appropriate.

In particular embodiments, computer system 600 includes a processor 602,memory 604, storage 606, an input/output (I/O) interface 608, acommunication interface 610, and a bus 612. Although this disclosuredescribes and illustrates a particular computer system having aparticular number of particular components in a particular arrangement,this disclosure contemplates any suitable computer system having anysuitable number of any suitable components in any suitable arrangement.

In particular embodiments, processor 602 includes hardware for executinginstructions, such as those making up a computer program. As an exampleand not by way of limitation, to execute instructions, processor 602 mayretrieve (or fetch) the instructions from an internal register, aninternal cache, memory 604, or storage 606; decode and execute them; andthen write one or more results to an internal register, an internalcache, memory 604, or storage 606. In particular embodiments, processor602 may include one or more internal caches for data, instructions, oraddresses. This disclosure contemplates processor 602 including anysuitable number of any suitable internal caches, where appropriate. Asan example and not by way of limitation, processor 602 may include oneor more instruction caches, one or more data caches, and one or moretranslation lookaside buffers (TLBs). Instructions in the instructioncaches may be copies of instructions in memory 604 or storage 606, andthe instruction caches may speed up retrieval of those instructions byprocessor 602. Data in the data caches may be copies of data in memory604 or storage 606 for instructions executing at processor 602 tooperate on; the results of previous instructions executed at processor602 for access by subsequent instructions executing at processor 602 orfor writing to memory 604 or storage 606; or other suitable data. Thedata caches may speed up read or write operations by processor 602. TheTLBs may speed up virtual-address translation for processor 602. Inparticular embodiments, processor 602 may include one or more internalregisters for data, instructions, or addresses. This disclosurecontemplates processor 602 including any suitable number of any suitableinternal registers, where appropriate. Where appropriate, processor 602may include one or more arithmetic logic units (ALUs); be a multi-coreprocessor; or include one or more processors 602. Although thisdisclosure describes and illustrates a particular processor, thisdisclosure contemplates any suitable processor.

In particular embodiments, memory 604 includes main memory for storinginstructions for processor 602 to execute or data for processor 602 tooperate on. As an example and not by way of limitation, computer system600 may load instructions from storage 606 or another source (such as,for example, another computer system 600) to memory 604. Processor 602may then load the instructions from memory 604 to an internal registeror internal cache. To execute the instructions, processor 602 mayretrieve the instructions from the internal register or internal cacheand decode them. During or after execution of the instructions,processor 602 may write one or more results (which may be intermediateor final results) to the internal register or internal cache. Processor602 may then write one or more of those results to memory 604. Inparticular embodiments, processor 602 executes only instructions in oneor more internal registers or internal caches or in memory 604 (asopposed to storage 606 or elsewhere) and operates only on data in one ormore internal registers or internal caches or in memory 604 (as opposedto storage 606 or elsewhere). One or more memory buses (which may eachinclude an address bus and a data bus) may couple processor 602 tomemory 604. Bus 612 may include one or more memory buses, as describedbelow. In particular embodiments, one or more memory management units(MMUs) reside between processor 602 and memory 604 and facilitateaccesses to memory 604 requested by processor 602. In particularembodiments, memory 604 includes random access memory (RAM). This RAMmay be volatile memory, where appropriate Where appropriate, this RAMmay be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, whereappropriate, this RAM may be single-ported or multi-ported RAM. Thisdisclosure contemplates any suitable RAM. Memory 604 may include one ormore memories 604, where appropriate. Although this disclosure describesand illustrates particular memory, this disclosure contemplates anysuitable memory.

In particular embodiments, storage 606 includes mass storage for data orinstructions. As an example and not by way of limitation, storage 606may include a hard disk drive (HDD), a floppy disk drive, flash memory,an optical disc, a magneto-optical disc, magnetic tape, or a UniversalSerial Bus (USB) drive or a combination of two or more of these. Storage606 may include removable or non-removable (or fixed) media, whereappropriate. Storage 606 may be internal or external to computer system600, where appropriate. In particular embodiments, storage 606 isnon-volatile, solid-state memory. In particular embodiments, storage 606includes read-only memory (ROM). Where appropriate, this ROM may bemask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM),electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM),or flash memory or a combination of two or more of these. Thisdisclosure contemplates mass storage 606 taking any suitable physicalform. Storage 606 may include one or more storage control unitsfacilitating communication between processor 602 and storage 606, whereappropriate. Where appropriate, storage 606 may include one or morestorages 606. Although this disclosure describes and illustratesparticular storage, this disclosure contemplates any suitable storage.

In particular embodiments, I/O interface 608 includes hardware,software, or both, providing one or more interfaces for communicationbetween computer system 600 and one or more I/O devices. Computer system600 may include one or more of these I/O devices, where appropriate. Oneor more of these I/O devices may enable communication between a personand computer system 600. As an example and not by way of limitation, anI/O device may include a keyboard, keypad, microphone, monitor, mouse,printer, scanner, speaker, still camera, stylus, tablet, touch screen,trackball, video camera, another suitable I/O device or a combination oftwo or more of these. An I/O device may include one or more sensors.This disclosure contemplates any suitable I/O devices and any suitableI/O interfaces 608 for them. Where appropriate, I/O interface 608 mayinclude one or more device or software drivers enabling processor 602 todrive one or more of these I/O devices. I/O interface 608 may includeone or more I/O interfaces 608, where appropriate. Although thisdisclosure describes and illustrates a particular I/O interface, thisdisclosure contemplates any suitable I/O interface.

In particular embodiments, communication interface 610 includeshardware, software, or both providing one or more interfaces forcommunication (such as, for example, packet-based communication) betweencomputer system 600 and one or more other computer systems 600 or one ormore networks. As an example and not by way of limitation, communicationinterface 610 may include a network interface controller (NIC) ornetwork adapter for communicating with an Ethernet or other wire-basednetwork or a wireless NIC (WNIC) or wireless adapter for communicatingwith a wireless network, such as a WI-FI network. This disclosurecontemplates any suitable network and any suitable communicationinterface 610 for it. As an example and not by way of limitation,computer system 600 may communicate with an ad hoc network, a personalarea network (PAN), a local area network (LAN), a wide area network(WAN), a metropolitan area network (MAN), or one or more portions of theInternet or a combination of two or more of these. One or more portionsof one or more of these networks may be wired or wireless. As anexample, computer system 600 may communicate with a wireless PAN (WPAN)(such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAXnetwork, a cellular telephone network (such as, for example, a GlobalSystem for Mobile Communications (GSM) network), or other suitablewireless network or a combination of two or more of these. Computersystem 600 may include any suitable communication interface 610 for anyof these networks, where appropriate. Communication interface 610 mayinclude one or more communication interfaces 610, where appropriate.Although this disclosure describes and illustrates a particularcommunication interface, this disclosure contemplates any suitablecommunication interface.

In particular embodiments, bus 612 includes hardware, software, or bothcoupling components of computer system 600 to each other. As an exampleand not by way of limitation, bus 612 may include an AcceleratedGraphics Port (AGP) or other graphics bus, an Enhanced Industry StandardArchitecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT)interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBANDinterconnect, a low-pin-count (LPC) bus, a memory bus, a Micro ChannelArchitecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, aPCI-Express (PCIe) bus, a serial advanced technology attachment (SATA)bus, a Video Electronics Standards Association local (VLB) bus, oranother suitable bus or a combination of two or more of these. Bus 612may include one or more buses 612, where appropriate. Although thisdisclosure describes and illustrates a particular bus, this disclosurecontemplates any suitable bus or interconnect.

Herein, a computer-readable non-transitory storage medium or media mayinclude one or more semiconductor-based or other integrated circuits(ICs) (such, as for example, field-programmable gate arrays (FPGAs) orapplication-specific ICs (ASICs)), hard disk drives (HDDs), hybrid harddrives (HHDs), optical discs, optical disc drives (ODDs),magneto-optical discs, magneto-optical drives, floppy diskettes, floppydisk drives (FDDs), magnetic tapes, solid-state drives (SSDs),RAM-drives, SECURE DIGITAL cards or drives, any other suitablecomputer-readable non-transitory storage media, or any suitablecombination of two or more of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes and illustrates respectiveembodiments herein as including particular components, elements,feature, functions, operations, or steps, any of these embodiments mayinclude any combination or permutation of any of the components,elements, features, functions, operations, or steps described orillustrated anywhere herein that a person having ordinary skill in theart would comprehend. Furthermore, reference in the appended claims toan apparatus or system or a component of an apparatus or system beingadapted to, arranged to, capable of, configured to, enabled to, operableto, or operative to perform a particular function encompasses thatapparatus, system, component, whether or not it or that particularfunction is activated, turned on, or unlocked, as long as thatapparatus, system, or component is so adapted, arranged, capable,configured, enabled, operable, or operative. Additionally, although thisdisclosure describes or illustrates particular embodiments as providingparticular advantages, particular embodiments may provide none, some, orall of these advantages.

What is claimed is:
 1. A system comprising: one or more processors, anda memory coupled to the processors comprising instructions executable bythe processors, the processors being operable when executing theinstructions to: send, through a communications network, sets of firstnotifications corresponding to a first notification type, wherein eachof the sets of first notifications has a corresponding volume and issent to a plurality of users, and the volumes corresponding to the setsof first notifications are different; send, through the communicationsnetwork, sets of second notifications corresponding to a secondnotification type, wherein each of the sets of second notifications hasa corresponding volume and is sent to a plurality of users, and thevolumes corresponding to the sets of second notifications are different;determine visitation impacts of the sets of first and secondnotifications of the first and second notification types, respectively,wherein each of the visitation impacts is associated with one of thesets of first and second notifications and the volume corresponding tothat set; train, based on the visitation impacts and the associatedvolumes, a machine-learning model to output assessments of the first andsecond notification types; determine, based on the assessments, adesired volume of a set of third notifications and a desirednotification type for the set of third notifications, wherein thedesired notification type is selected from the first notification typeor the second notification type; and send, through the communicationsnetwork, the set of third notifications.
 2. The system of claim 1,wherein the processors are further operable when executing theinstructions to: receive a request to send a plurality of thirdnotifications to a plurality of target users, wherein the set of thirdnotifications is a subset of the plurality of third notifications; andselect a set of target users from the plurality of target users based onthe desired volume of the set of third notifications; wherein the set ofthird notifications are sent to the selected set of target users.
 3. Thesystem of claim 2, wherein the processors are further operable whenexecuting the instructions to: derive, based on the machine-learningmodel, a characteristic value associated with a desired usercharacteristic for the set of third notifications, wherein the set oftarget users are selected based on the desired user characteristic. 4.The system of claim 1, wherein the processors are further operable whenexecuting the instructions to: derive, based on the visitation impacts,a curve of visitation impact versus volume for the first notificationtype or the second notification type, wherein at least one of thedesired volume or the desired notification type is determined based onthe curve.
 5. The system of claim 4, wherein the desired volume isdetermined based on an application of a threshold visitation impactagainst the curve, wherein the threshold visitation impact is based on acost of sending notifications of the first notification type or thesecond notification type.
 6. The system of claim 1, wherein thevisitation impacts comprise a user rate of conversion responsive to anotification, a user rate of activity responsive to the notification, ora number of users that have accessed a social-networking system in atime period responsive to the notification.
 7. The system of claim 1,wherein each of the first notification type and the second notificationtype corresponds to Simple Message Service (SMS) notifications,Multimedia Messaging Service (MMS) notifications, e-mails, or pushnotifications.
 8. The system of claim 1, wherein the first notificationtype comprises a social-network post notification, a social-network tagnotification, a social-network comment notification, or a social-networkphoto notification.
 9. The system of claim 3, wherein the desired usercharacteristic comprises a user demographic, a user usage pattern, auser device type, a social-network friend count, or a social-networkage.
 10. One or more computer-readable non-transitory storage mediaembodying software that is operable when executed to: send, through acommunications network, sets of first notifications corresponding to afirst notification type, wherein each of the sets of first notificationshas a corresponding volume and is sent to a plurality of users, and thevolumes corresponding to the sets of first notifications are different;send, through the communications network, sets of second notificationscorresponding to a second notification type, wherein each of the sets ofsecond notifications has a corresponding volume and is sent to aplurality of users, and the volumes corresponding to the sets of secondnotifications are different; determine visitation impacts of the sets offirst and second notifications of the first and second notificationtypes, respectively, wherein each of the visitation impacts isassociated with one of the sets of first and second notifications andthe volume corresponding to that set; train, based on the visitationimpacts and the associated volumes, a machine-learning model to outputassessments of the first and second notification types; determine, basedon the assessments, a desired volume of a set of third notifications anda desired notification type for the set of third notifications, whereinthe desired notification type is selected from the first notificationtype or the second notification type; and send, through thecommunications network, the set of third notifications.
 11. The media ofclaim 10, wherein the software is further operable when executed to:derive, based on the machine-learning model, a characteristic valueassociated with a desired user characteristic for the set of thirdnotifications, wherein the set of target users are selected based on thedesired user characteristic.
 12. The media of claim 10, wherein thesoftware is further operable when executed to: derive, based on thevisitation impacts, a curve of visitation impact versus volume for thefirst notification type or the second notification type, wherein atleast one of the desired volume or the desired notification type isdetermined based on the curve.
 13. The media of claim 12, wherein thedesired volume is determined based on an application of a thresholdvisitation impact against the curve, wherein the threshold visitationimpact is based on a cost of sending notifications of the firstnotification type or the second notification type.
 14. The media ofclaim 10, wherein the visitation impacts comprise a user rate ofconversion responsive to a notification, a user rate of activityresponsive to the notification, or a number of users that have accesseda social-networking system in a time period responsive to thenotification.
 15. The media of claim 10, wherein each of the firstnotification type and the second notification type corresponds to SimpleMessage Service (SMS) notifications, Multimedia Messaging Service (MMS)notifications, e-mails, or push notifications.
 16. Acomputer-implemented method comprising, by one or more computingdevices: sending, through a communications network, sets of firstnotifications corresponding to a first notification type, wherein eachof the sets of first notifications has a corresponding volume and issent to a plurality of users, and the volumes corresponding to the setsof first notifications are different; sending, through thecommunications network, sets of second notifications corresponding to asecond notification type, wherein each of the sets of secondnotifications has a corresponding volume and is sent to a plurality ofusers, and the volumes corresponding to the sets of second notificationsare different; determining visitation impacts of the sets of first andsecond notifications of the first and second notification types,respectively, wherein each of the visitation impacts is associated withone of the sets of first and second notifications and the volumecorresponding to that set; training, based on the visitation impacts andthe associated volumes, a machine-learning model to output assessmentsof the first and second notification types; determining, based on theassessments, a desired volume of a set of third notifications and adesired notification type for the set of third notifications, whereinthe desired notification type is selected from the first notificationtype or the second notification type; and sending, through thecommunications network, the set of third notifications.
 17. The methodof claim 16, further comprising: receiving a request to send a pluralityof third notifications to a plurality of target users, wherein the setof third notifications is a subset of the plurality of thirdnotifications; and selecting a set of target users from the plurality oftarget users based on the desired volume of the set of thirdnotifications; wherein the set of third notifications are sent to theselected set of target users.
 18. The method of claim 17, furthercomprising: deriving, based on the machine-learning model, acharacteristic value associated with a desired user characteristic forthe set of third notifications, wherein the set of target users areselected based on the desired user characteristic.
 19. The method ofclaim 16, further comprising: deriving, based on the visitation impacts,a curve of visitation versus volume for the first notification type orthe second notification type, wherein at least one of the desired volumeor the desired notification type is determined based on the curve. 20.The method of claim 19, wherein the desired volume is determined basedon an application of a threshold visitation impact against the curve,wherein the threshold visitation impact is based on a cost of sendingnotifications of the first notification type or the second notificationtype.